Estimating Image Motion in Layers: The “Skin and Bones” Model

Estimating Image Motion in Layers: The \Skin and Bones" Model Xuan Ju Doctor of Philosophy Graduate Department of Computer Science University of Toronto 1998 This thesis addresses the problem of recovering a locally layered representation of image motion. We develop the \Skin and Bones" model for estimating optical ow that strikes a balance between the exibility of regularization techniques and the robustness and accuracy of area-based regression techniques. The approach assumes that image motion can be represented by an a ne ow model within local image patches. Since some image regions may not have su cient information to estimate an a ne motion model robustly, we de ne a spatial smoothness constraint on the a ne ow parameters of neighboring patches. We refer to this as a \Skin and Bones" model in which the a ne patches can be thought of as rigid patches of \bone" connected by a exible \skin." Since local image patches may contain multiple motions we use a layered representation for the a ne bones. With the possibility of multiple motions at a given point, standard regularization schemes cannot be used to smooth the multiple sets of a ne parameters. We therefore develop a new framework for regularization with transparency that can applied to produce a smoothed layered motion representation. The motion estimation problem, with layered locally a ne patches and transparent regularization, is formulated as an objective function that is minimized using a variant of the Expectation-Maximization (EM) algorithm. In addition, we also formulate spatial and temporal smoothness constraints on the EM ownership weights at the pixel level. This formulation ts naturally into the EM framework. We also exploit an incremental revision process to estimate the number of layers in each patch using the Minimum